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1 Introduction 



For K/F a quadratic extension of p-adic fields, let a be the conjugation relative to this extension, 
and rj K ip be the character of F* with kernel norms of K* . 

If 7T is a smooth irreducible representation of GL(n, K), and x a character of F* , the dimension of 
the space of linear forms on its space, which transform by x under GL(n, F) (with respect to the 
action [(L,g) h- > L o ir(g)]) , is known to be at most one (Proposition 11, [P]). One says that n is 
X-distinguished if this dimension is one, and says that it is distinguished if it is 1-distinguished. 
In this article, we give a description of distinguished principal series representations of GL(n,K), 
The result (Theorem 13. 2[) is that the the irreducible distinguished representations of the principal 
series of GL(n, K) are (up to isomorphism) those unitarily induced from a character x = (Xi> — i Xn) 
of the maximal torus of diagonal matrices, such that there exists r < n/2, for which xf+i = X» _1 
for i = 1, 3, .., 2r — 1, and x%\f* = 1 f° r * > 2r. For the quadratic extension C/R, it is known (cf.[F]) 
that the analogous result is true for tempered representations. 

For n > 3, this gives a counter-example (Corollarv l3.ip to a conjecture of Jacquet (Conjecture 1 in 
[A]). This conjecture states that an irreducible representation w of GL(n, K) with central character 
trivial on F* is isomorphic to fr' 7 if and only if it is distinguished or ^/^-distinguished (where 
iJk/f is the character of order 2 of F*, attached by local class field theory to the extension K/F). 
For discrete series representations, the conjecture is verified, it was proved in [KJ. 
Unitary irreducible distinguished principal series representations of GL(2, K) were described in [HJ, 
and the general case of distinguished irreducible principal series representations of GL(2, K) was 
treated in |F-H] . We use this occasion to give a different proof of the result for GL(2, K) than the 
one in |F-H| . To do this, in Theorems 14.11 and 14.21 we extend a criterion of Hakim (th.4.1, [H]) 
characterising smooth unitary irreducible distinguished representations of GL(2, K) in terms of 7 
factors at 1/2, to all smooth irreducible distinguished representations of GL(2, K). 



2 Preliminaries 

Let <j) be a group automorphism, and x an element of the group, we sometimes note instead of 
4>{x), and x~4> the inverse of x^ . If (p = x 1— > h~ 1 xh for h in the group, then x h designs x® . 
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Let G be a locally compact totally disconnected group, H a closed subgroup of G. 
We note Aq the module of G, given by the relation do(gx) — Ac(g)dG(x), for a right Haar measure 
g?g on G. 

Let X be a locally closed subspace of G, with H.X C X. If V is a complex vector space, we note 
D(X, V) the space of smooth I^-valued functions on X with compact support (if V — C, we simply 
note it D(X)). 

Let p be a a smooth representation of in a complex vector space V p , we note D(H\X, p,V p ) 
the space of smooth V^-valued functions / on X, with compact support modulo H, which verify 
f(hx) — p{h)f(x) for h G £f and a; G X (if p is a character, we note it D(H\X, p)). 
We note ind ( f I {p) the representation by right translations of G in 
D(H\G,(A G /A H y/ 2 p,V p ). 

Let f be a non archimedean local field of characteristic zero, and K a quadratic extension of 
F. We have K = F{5) with S 2 in F*. 

We note | \k and | |f the modules of K and i* 1 respectively. 

We note a the non trivial element of the Galois group G(K/F) of K over F, and we use the same 
letter to design for its action on M n (K). 

We note N K j F the norm of the extension K/F and we note T)k/f the nontrivial character of F* 
which is trivial on N K / F (K*). 

Whenever G is an algebraic group defined over F, we note G(K) its if-points and G(F) its i^-points. 

The group GL{n) will be noted G n , its standard Borel subgroup will be noted B n , its unipotent 
radical U n , and the standard maximal split torus of diagonal matrices T n . 

We note S the space of matrices M in G n (K) satisfying MM a = 1 . 

Everything in this paragraph is more or less contained in (Flj . we give detailed proofs here for 
convenience of the reader. 

Proposition 2.1. (J^/, ch.10, prop. 3) 

We have a homeomorphism between G n {K)/G n (F) and S given by the map S n : g i— > g° ' g~ l . 

Proposition 2.2. For its natural action on S, each orbit of B n (K) contains one and only one 
element of & n of order 2 or 1. 

Proof. We begin with the following: 

Lemma 2.1. Let w be an element of & n C G n {K) of order at most 2. 

Let 9' be the involution of T n (K) given by t i— > w~ 1 t a w, then any t G T n (K) with td'(t) = 1 is 
of the form a/9' (a) for some a G T n (K). 

Proof of Lemma [2~J[ There exists r < n/2 such that up to conjugacy, w is 
(l,2)(3,4)...(2r-l,2r). 
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We write t 



zi 



Z2r+1 



, hence for i < r, we have Zia{z' i ) = 1, 



V z n J 

and Zj<j(zj) = 1 for j > 2r + 1. 

Hilbert's Theorem 90 asserts that each Zj, j > 2r + 1 is of the form Uj-2r/°'( u j-2r), for some 

/ ^1 \ 
1 



We then take a 



"l 



V 



□ 



U n -2r / 



Lemma 2.2. Let N be an algebraic connected unipotent group over K . Let 9 be an involutive 
automorphism of N(K). If x G N(K), verifies x0(x) = ljv then, there is a e N such that 
x = 9(a~ 1 )a. 

Proof of Lemma POl The group N(K) has a composition series In = Nq C Ni C ...iV n _i C iV„ = 
N(K), such that each quotient N i+ i/Ni is isomorphic to (if, +), and each commutator subgroup 
[N,Ni + i] is a subgroup of iV*. 
Now we prove the Lemma by induction on n: 

If n = 1, then N(K) is isomorphic to (K, +), one concludes taking a = x/2. 
n i y n -\- 1 : 

suppose the Lemma is true for every N(K) of length n. 

Let N(K) be of length n + 1, we note x the class of x in N(K)/Ni. 

By induction hypothesis, one gets that there exists an element in h e Ni, and an element u in 
iV(lf) such that x = 9{u~ v )uh. 

Here h lies in the center of N(K), because [N(K), Ni] = 

As x9(x) — 1, we get h9(h) = 1. By induction hypothesis again, we get h = 9(b~ 1 )b for b G N\. 
We then take a = ub. □ 



We get back to the proof of the Proposition 12.2 



For w in S„, one notes U w the subgroup of U n generated by the elementary subgroups U a , with 
a positive, and wa negative, and Uj the subgroup of U n generated by the elementary subgroups 
U a , with a positive, and wa positive. Then U n = U W 'U W . 

Let s be in S. According to Bruhat's decomposition, there is w in 6„, and a in T n (K), m in U n (K) 
and n2 + in U w , such that s = niawri2 + , with unicity of the decomposition. 
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Then s = s~ CT = ii2 + " w~ x aT a n\~ a . 
Thus we have aw = (aw)~ a , i.e. w 2 = 1 and a w = a~ a . 

Now we write ni~ a = u~u + with u~ S Uj and u + € U w , comparing s and s _<T , u + must be equal 
to n 2 + . 

Hence s = n\awu~ n^" 7 , thus we suppose s = awn, with n in f/^- 

From s = s~ a , one has the relation awn^aw)^ 1 — n~ a , applying a on each side, this becomes 
(aw)~ 1 n a 'aw = n . 

But '. u i — ► (aw)~ 1 u a aw is an involutive automorphism of C/^, hence from Lemma \2~2\ there is u' 
in such that n = 8(u~ 1 )u. 

This gives s = u~ a awu, so that we suppose s = aw. Again wa a w = a~ x , and applying Lemma I2TTI 
to 6' : x \— * wx a w, we deduce that a is of the form yO'd/^ 1 ), and s ~ ywy~ a . □ 

Let u be the element ( \ x ) of M 2 (if) ; one has 5 2 (u) — ( ® ?. ] (cf . Proposition 12. ip . 



v 1 5 J Wi "™ - ^ l / 

We notice for further use (cf. proof of Proposition 3.1), that if we note T the subgroup j ( q ) £ ^(if)^ G if* ^, 



thenu- 1 f« = r=|f J ^ )eG 2 (F)|z,y G F}. 



For r < n/2, one notes C/ r the n x n matrix given by the following block decomposition: 
/ u \ 



u 

In-2r / 

If w is an element of 6 n naturally injected in G n (K), one notes U™ = w^UrW. 

Corollary 2.1. The elements U™ for < r < n/2, and w G & n give a complete set of representa- 
tives of classes of B n (K)\G n (K) / G n (F) . 

Let G n — Yiwee B n wB n be the Bruhat decomposition of G n . We call a double-class BwB a 
Bruhat cell. 

Lemma 2.3. One can order the Bruhat cells G\, Ci,..., G n \ so that for every 1 < i < n\, the cell 
Ci is closed in G n — TJfc=i 

Proof. Choose C\ — B n . It is closed in G n . Now let W2 be an element of 6„ — Id, with minimal 



length. Then from 8.5.5. of |Sp| , one has that the Bruhat cell BW2B is closed in G n — B n with 
respect to the Zariski topology, hence for the p-adic topology, we call it C2. We conclude by 
repeating this process. □ 

Corollary 2.2. One can order the classes Ax,..,, A t of B n (K)\G n (K) / G n (F) , so that Ai is closed 
mG„(if)-IJtU.- 

Proof. From the proof of Proposition ^. 2\ we know that if C is a Bruhat cell of G n , then S n R C is ei- 
ther empty, or it corresponds through the homemorphism S n to a class A of B n (K)\G n (K) / G n (F) . 
The conclusion follows the preceeding Lemma. □ 

Corollary 2.3. Each Ai is locally closed in G n (K) for the Zariski topology. 
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We will also need the following Lemma: 



Lemma 2.4. Let G, if ,X , and (p, V p ) be as in the beginning of the section, the map <f> from 
D(X) ® V p to D(H\X, p, V p ) defined by $ : / ® v h-> (x J h f(hx)p(h~ 1 )vdh) is surjective. 

Proof. Let v € V p , U an open subset of G that intersects X, small enough for h i— » p(h)v to be 
trivial on if n E/T7 -1 . 

Let /' be the function with support in if (X n U) defined by hx i— > p(h)v. 
Such functions generate D(H\X, p, V p ) as a vector space. 

Now let / be the function of D(X, V p ) defined by x i— > l[/ n x(a;)w, then <&(/) is a multiple of /'. 
But for a: in (7 fl X, $(/)(x) = J H / (hx)dh because h \— > p(h)v is trivial on if (~l UU^ 1 , plus 

ft M f{hx) is a positive function that multiplies u, and /(a;) = V, so F(f)(x) is u multiplied by a 
strictly positive scalar. □ 

Corollary 2.4. Lei Y be a closed subset of X , if -stable, then the restriction map from D{H\X, p, V p ) 
to D(H\Y, p,V p ) is surjective. 

Proof. This is a consequence of the known surjectivity of the restriction map from D(X) to D(Y), 
which implies the surjectivity of the restriction from D(X, V p ) to D(Y, V p ) and of the commutativity 
of the diagram: 



D{H\X,p) -» D(H\Y,p) 

3 Distinguished principal series 

If 7T is a smooth representation of G n (K) of space V n , and \ is a character of F* , we say that n is 
X-distinguished if there exists on Kr a nonzero linear form L such that L(w(g)v) — x(det(g))L(v) 
whenever g is in G n (F) and v belongs to V„, If x ls trivial, we simply say that ir is distinguished. 

We first recall the following: 

Theorem 3.1. (JPj. Proposition 12) 

Let 7r be a smooth irreducible distinguished representation of G n (K), then ir a ~ w. 

Let xi, Xn be n characters of K* , with none of their quotients equal to | \k- We note x the 



We note 7r(x) the representation of G n (K) by right translation on the space of functions D(B n (K)\G n (K), Ab^x). 
This representation is smooth, irreducible and called the principal series attached to x- 
If 7T is a smooth representation of G n (K), we note pi its smooth contragredient. 

We will need the following Lemma: 

Lemma 3.1. (Proposition 26 in \F1^ ) Let fh — (mi, . . . , mi) be a partition of a positive integer m, 
let Pfh be the corresponding standard parabolic subgroup, and for each 1 < i < I , let iTi be a smooth 



D(X) 
1$ 



D(Y) 
1$ 



□ 
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Q / J£ \ 1/2 

distinguished representation of G mi (K), then Hi x • • • x 717 = ind p m ^ K ^ , (A p _ (^(tti <8> ■ • • ® tt;)) is 

We now come to the principal results: 

Proposition 3.1. Let x = {Xii ■ ■ ■ Jn) be a character ofT n {K), suppose that the principal series 
representation ir(x) * s distinguished, there exists a re-ordering of the Xi ' s > an d r < such that 
Xt+i = XC 1 for i = 1, 3, .., 2r - 1, and that Xi\F* = 1 for i > 2r. 

Proof. We write B = B n (K), G — G n (K). We have from Corollary 12.21 and 12 .41 the following exact 
sequence of smooth G„(i 7 ')-modules: 

D(B\G - A u A B 1/2 X ) - D(B\G, A B 1/2 X ) -> D(B\A U A" 1/2 X ). 

— 1/2 

Hence there is a non zero distinguished linear form either on D(B\Ai, A B x), or on 
D(B\G-A 1 ,A~ 1/2 X ). 

In the second case we have the exact sequence 

D(B\G -A X U A 2 ,A B 1/2 X ) - D(B\G - A u A B 1/2 X ) - D(B\^ 2j A B 1/2 X ). 

Repeating the process, we deduce the existence of a non zero distinguished linar form on one of the 

2/2 

spaces D(B\Ai,A B X ). 

From Corollary 12.11 we choose w in S n and r < n/2 such that A l = BU™G n (F). The applica- 

^ /2 

tion f 1 ► 1 ► f(UrX)] gives an isomorphism of G„(F)-modules between D(B\A i: A B X ) and 

D{U- W BU™ n G n (F)\G n (F), A'x!) where A'(ai) = A.~ 1/i (U?xU^ w ) and ^(ss) = x(U™xlJ- w ). 
Now there exists a nonzero G„(-F)-invariant linear form on 

D(JJ- W BU™ n Gn{F)\G n {F), AY) if and only if AY is equal to the inverse of the module 
of U~ W BU™ n G„(F) (cf. |B-H| . ch.l, prop.3.4). From this we deduce that X ' is positive on 
U- W BU™ n G n (F) or equivalently x is positive on £ n U™G n {F)lJ- w . 
Let T r be the F-torus of matrices of the form 

/ zi \ 

Xi 

V *t) 

with 2r + t ~ n, Zi e K* , Xt £ F* , then one has f r w C B n U™G n {F)lJ- w , so that x must be 
positive on T r . 

We remark that if x is unitary, then x is trivial on T^, and 7r(x) is of the desired form. 

For the general case, we deduce from Theorem l3.il that there exists three integers p > 0, <? > 0, s > 

such that up to reordering, we have X2i — Xii'-i f° r — * — P> we have X2p+ki F , = 1 for 1 < k < q 
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and these X2 P +k's are different (so that X2 P +k ^ X2 P +k' f° r k ^ k'), and X2p+q+j\p* = Vk/f for 
1 < j < s , these X2p+q+j's being different. 

We note fik = X2 P +k for g > fc > 1, and i/£ = X2 P + g +k' for s > fc'_> 1. 

We show that if such a character x is positive on a conjugate of T r by an element of S n , then s = 0. 
Supose V\ appears, then either v\ is positive on F* , but that is not possible, or it is coupled with 
another Xi, an d (vi, Xi) ls positive on elements (z, z a ), for z in K* . 

Suppose Xi = f° r some J 7^ 1; then (vi,Xi) ls unitary, so it must be trivial on couples (z, z a ), 
which implies v\ — Vj~° — Vj, which is absurd. 

The character Xi cannot be of the form fij, because it would imply vup* = 1. 

The last case is i < 2p, then v\~ a = v\ must be the unitary part of Xi because of the positivity of 

(yi, Xi) ° n the couples (z, z a ). 

But Xi~° a l so appears and is not trivial on F*, hence must be coupled with another character Xj 
with j < 2p and j ^ i, such that (xi _<T , Xj) ls positive on the elements (z, z a ), for z in K*, which 
implies that Xj has unitary part vi~ a = v\. The character Xj cannot be a fik because of its unitary 
part. 

If it is a Xk with k < 2p, we consider again Xk~ a ■ 

But repeating the process lengthily enough, we can suppose that Xj ls °f the form i>k, for k ^ 1. 
Taking unitary parts, we see that Vk = = v\, which is in contradiction with the fact that all 
iVs are different. We conclude that s = 0. □ 

Theorem 3.2. Let x — (xi> ■ ■ ■ ;X«) ^ e a character ofT n (K), the principal series representation 
7r(x) is distinguished if and only if there exists r < n/2, such that xf+i = X^ 1 f or i = 1, 3, .., 2r — 1, 
and that Xi\F* = 1 f or * > %r. 

Proof. There is one implication left. 

Suppose x is °f the desired form, then 7r(x) is parabolically (unitarily) induced from representations 
of the type 7r(xi,Xi~ cr ) of G2(K), and distinguished characters of K*. 

Hence, because of Lemma 13.11 the Theorem will be proved if we know that the representations 
7r (Xi)Xi~ <T ) are distinguished, but this is Corollary 14. II of the next paragraph. □ 

This gives a counter-example to a conjecture of Jacquet (conjecture 1 in (Aj), asserting that if 
an irreducible admissible representation it of G n (K) verifies that n is isomorphic to 7r CT , then it is 
distinguished if n is odd, and it is distinguished or /^/^-distinguished if n is even. 

Corollary 3.1. For n > 3, there exist smooth irreducible representations ir of G n (K), with central 
character trivial on F* , that are neither distinguished, nor 
Vk/f -distinguished, but verify that ir is isomorphic to Tr a . 

Proof. Take xi ; ■ ■ • > Xn, all different, such that Xi|F* = X2|f* = Vk/f, an d Xjip* = 1 f° r 3 < j < n. 
Because each Xi has trivial restriction to N K / F (K*), it is equal to x7° ' > hence tt is isomorphic to 7r CT . 
Another consequence is that if k and I are two different integers between 1 and n, then Xfc ^ Xi° \ 
because we supposed the Xi' s ah different. 

Then it follows from Theorem 13.21 that tt = 7r(xi, ■ ■ ■ , x«) is neither distinguished, nor Vk/f~ 
distinguished, but clearly, the central character of 7r is trivial on F* and tt is isomorphic to 7r CT . □ 



7 



4 Distinction and gamma factors for GL(2) 



As said in the introduction, in this section we generalize to smooth infinite dimensional irreducible 
representations of G2(K) a criterion of Hakim (cf. [H], Theorem 4.1) characterising smooth unitary 
irreducible distinguished representations of G%(K), In proof of Theorem 4.1 of [H], Hakim deals 
with unitary representations so that the integrals of Kirillov functions on F* with respect to a Haar 
measure of F* converge. We skip the convergence problems using Proposition 2.9 of chapter 1 of 

HH 

( (X x \ 

We note M(K) the mirabolic subgroup of G2(K) of matrices of the form I n 1 ) with a in 

K* and x in K, and M(F) its intersection with G2(F). We note w the matrix 

Let 7r be a smooth infinite dimensional irreducible representation of G2(K), it is known that it is 
generic (cf.[Z] for example). Let K(jT : ip) be its Kirillov model corresponding to ip ( [J-L| . th. 2.13), 
it contains the subspace D(K*) of functions with compact support on the group K* . If <p belongs 

to K(n,ip), and x belongs to K, then cj>— ^ ^ g ^ ^ <t> belongs to D(K*) ( [J-L| . prop. 2. 9, ch.l), 

from this follows that K(tt, ip) = D(K*) + it(w)D(K*). 

We now recall a consequence of the functional equation at 1/2 for Kirillov representations (cf. 
[B], section 4.7). 

Forall <p m K(n, ip) and \ character of K* , we have whenever both sides converge absolutely: 

J K . n(w)(t)(x)(c^xy 1 (x)d* x = 7(tt ® x,^P) f K . <t>(x)x(x)d* x (1) 
where d*x is a Haar measure on K* , and is the central character of tt. 




Theorem 4.1. Let tt be a smooth irreducible representation of G2(K) of infinite dimension with 
central character trivial on F* , and ip a nontrivial character of K trivial on F. If 7(71-® x, = 1 
for every character x of K* trivial on F* , then tt is distinguished. 

Proof. In fact, using a Fourier inversion in functional equation[TJand the change of variable x >— > x _1 , 
we deduce that forall 4> in D(K*) n tt(w)D(K*), we have 

Ck{x) [ TT{w)(j>{tx- 1 )d*t= [ <f>(tx)d*t 

J F* J F* 

(d*t is a Haar measure on F*) which for x = 1 gives 

f TT(w)(f>{t)d*t= [ <f)(t)d*t. 
J F* J F* 

Now we define on K(tt, ip) a linear form A by: 

\{<h. +n(w)<p 2 ) = [ Mt)d*t+ [ Mt)d*t 

J F* J F* 
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for <pi and 02 in D(K*), which is well defined because of the previous equality and the fact that 

K(w,ip)=D(K*)+n(w)D(K*), 



It is clear that A is w-invariant. As the central character of n is trivial on F* , A is also F*- 
invariant. Because GL 2 (F) is generated by M(F), its center, and w, it remains to show that A is 
M{F) -invariant . 

Since ip is trivial on F, one has if G D{K*) and m G M(F) the equality A(7r(m)0) = A(0) . 
Now if — tt(w)4>2 G n(w)D(K* ), and if a belongs to F* , then n ( ° ^ J n(w)(f)2 = n(w)TT y J ^ 

7r(w)7r ^ q ^ J 02 because the central character of n is trivial on F* , and A(7r ^ q ^ 
A(0). 

/ 1 x A fix 
If x G -F, then ir ( ^ I 0—0 is a function in D (K*), which vanishes on F* , hence \ir( ( ^ 

0)=O. 

Eventually A is M(F)-invariant, hence G2(F)-invariant, it is clear that its restriction to D(K*) 
is non zero. □ 

Corollary 4.1. Let /x be a character of K* , then 7r(/i,// _<T ) is distinguished. 

Proof, indeed, first we notice that the central character [i[i~ a of 7r(//, A* -<t ) is trivial on F*. 
Now let x be a character otK*/F*, then 7(7r(/x, ^ _<t )<8)Xi = 7(^X; % l J )l{t L ~' 7 Xi "0) = li^Xi 1 P)l(^~ 1 X.' 7 ■> 4'°'), 
and as = 1 and X|f* = 1, one has V" 7 = V" -1 an d X a = X i so that 7(7r(x ; X~ ct )i VO = 
7(^X7 V , )7(m~ 1 X _1 j = 1- The conclusion falls from Proposition 14. II □ 

Assuming Theorem 1.2 of |A-G| . the converse of Theorem 14. II is also true: 



Theorem 4.2. Let ir be a smooth irreducible representation of infinite dimension of G%(K) with 
central character trivial on F* and ip a non trivial character of K/F, it is distinguished if and only 
if 7(77 ® Xj = 1 f or every character \ of K* trivial on F* . 

Proof. It suffices to show that if ir is a smooth irreducible distinguished representation of infinite 
dimension of G2(K), and ip a non trivial character of K/F, then 7(71", -0) = 1. Suppose A is a non 
zero G2(F)-invariant linear form on K(n, tp), it is shown in the proof of the corollary of Proposition 
3.3 in [H], that its restriction to D(F*) must be a multiple of the Haar measure on F* . Hence for 
any function in D{K*) n ir(w)D(K*), we must have J F , cj>(t)d*t = j F , ir(w)(f)(t)d*t. 
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From this one deduces that for any function in D(K*) fl ir(w)D(K*): 
Tr(w)(j)(x)c~ 1 (x)d* x = / e~ (a) / ir(w)4>(ta)d*tda 

K* JK*/F* JF* 



c-\a) [ tt( ° J )n(w)cf>(t)d*tda 

K* /F* J w 1 1 



K* IF 
f 

K*/F* JF 



Q 

7r(w)c 7r (a)7r( Q _^ )4>(t)d*tda 



/ tt( ° J )4>(t)d*tda 



^(ta-^dHda 

K* IF* JF* 



(f>(ta)d*tda 



K* IF* JF* 

4>(x)d*x 



K* 

This implies that either 7(71-,^) is equal to one, or j K ,<j){x)d*x is equal to zero on D{K*) n 
ir(w)D(K*). In the second case, we could define two independant fsT*-invariant linear forms on 
K(ir,ip) = D(K*) + ir(w)D(K*), given by 0i + ir(w)<f)2 1— » J^-* 0i(x)d*x, and </>i + ir(w)(f)2 1— > 
JV. 4>2{x)d*x. This would contradict Theorem 1.2 of |A-G] . □ 
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